Specifically, we summarize our contributions into the following points.

We explore online learning in episodic Markov decision processes on non-stationary environments (changing losses and probability transitions).

Constrained decision-making problems are pervasive across various disciplines.

Therefore, developing grid analysis algorithms is important for supporting reliable operations.

Lemma 3. Consider the ReLU activation The proof of Theorem 1 is given below. The inequality 3 uses strictly monotone property of p () . Code is available at this link. The neural networks are updated using Adam with learning rate initializes at 0.035 and All of them have no communication constraints. The training time is shown in Table 1.